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Abstract 

In this short paper, we consider a quadruple (fl, ,1!}, fi), where jz/ is a cr-algebra 
of subsets of n, and is a measurable bijection from into itself that preserves the 
measure fi. For each B £ £/, we consider the measure fXB obtained by taking cycles 
(excursions) of iterates of i? from B. We then derive a relation for /is that involves 
the forward and backward hitting times of B by the trajectory {'&"'uj, n € Z) at a point 
S il. Although classical in appearance, its use in obtaining uniqueness of invariant 
measures of various stochastic models seems to be new. We apply the concept to 
countable Markov chains and Harris processes. 
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1 Introduction 



This paper was initiated from the following question. It is classical that, for a Markov 
chain (X„,n > 0) with a countable state space S that possesses a positive recurrent state 
b € S, there is at least one invariant probability measure tt^^'^ on S which is defined by the 
usual "cycle formula": 

, tb-i 
^^^^ ^0 

where tf, is the first return time to h. To ensure that 7r('') is the only invariant probability 
measure we need, in addition, to ensure that the only positive recurrent states are those 
that communicate with h (this holds, for instance, if the chain is irreducible). There are 
several proofs of uniqueness, ranging from analytic (e.g. by means of the Perron-Frobenius 
theorem which itself can be proved in a number of ways-see, e.g. Lind and Marcus (1995) 
for a geometric proof) to probabilistic (e.g. by means of applying the Doeblin coupling 
construction: this requires, in addition, aperiodicity-see, e.g. Thorisson (2000)). The 
question we posed is whether there is a way to prove uniqueness directly from the way 
that vrW is constructed by the cycle formula. If so, can we do this for Markov chains in 
a general state space? And finally, how "Markovian" is the proof of uniqueness (can the 
"local" character of definition of vr;, be extended to other processes)? 
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In answering the question, we abstracted the problem and hfted it to a general mea- 
surable space (ri, i/) endowed with a measurable bijective transformation ■!? that preserves 
some measure fi. The point of view appears to be new, although the tools used below are 
quite natural in Ergodic Theory and in the construction of Palm Probabilities. The origin 
of these tools can be traced, as far as we can tell, to a paper by Kac (1947). In Section [2] 
we define, for each B G £/, the forwards and backwards hitting times of B by the iterates 
of ■!? (called Tb,Tb, respectively) and the measure 

. Tb-1 
•^^ n=0 

Theorem [T] states the basic formula of interest: 

HsiA) = ii{A,Tb < oo). 

It can be read as: on the event that B has been visited in the past at least once, the 
measures ^xb and fi coincide. Thus, if n{B) > 0, Poincare's recurrence lemma (recalled as 
Lemma [1]), fis = fJ- for all B. In Section [3l we consider a Markov chain on a countable 
set S. Assuming irreducibility and positive recurrence, the previous observation immedi- 
ately yields a unique probability measure tt on S" such that vrP = tt, which answers the 
original question. Finally, in Section HI we consider a Harris chain and show uniqueness of 
the invariant probability measure constructed by means of cycles away from a recurrent 
regeneration set R. 



2 The master formula 



Let be a measurable space and : — > a measurable bijection. For A,B & 

define the following functions: 

Tb = Tb{lo; 1?) := inf{n € N : G (la) 

Mb{A)=Mb{A,uj;^):= ^ l(i?"u; G ^), (lb) 

0<n<Ts(a;;i?) 

stressing that both take values in N U {+00} := {1, 2, . . .} U {+00}, where inf = +00. 
The definition of Mb (A) requires giving a meaning to the quantity t?^^. We let Qb = 
{Tb < 00}, and define t?^^ : Qb ^ by 

(i?^«)(u;) :=i?^^('^)(u;), w G J^b- 

On Q — Qb, we define i?"^^ rather arbitrarily, e.g. by letting it to be the identity on it. We 
can easily see that 'd'^^ is invertible with (t?'^^) = , and where is defined in 

a similar way. We shall also need (jlap - ()lbp when using 1?^^ in place of t?: 

Tb = Tb{uj; ^^^) := inf{n G N : G B} 

Mb{A) = Mb{A, oj- i?-1) := l(t?-"u; G A). 

0<ra<Ti3(a;;i9-i) 
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The interpretation is that Mb{A) evaluated at to is the number of times the forward 
trajectory {u, "duj, •d'^u, . . .) visits the set A up to (and not including) the time it visits the 
set B. Similarly, Mb{A) refers to the backward trajectory (w, '&~^oj, '&~^u, . . .). There is a 
certain "duality" between Mb{A) and Tg on one hand and Mb {A) and Tg on the other, 
once we integrate against an invariant measure. We discuss this next. Recall first the 
following standard lemma: 

Lemma 1 (Poincare recurrence). // the measure ji on is preserved by -d then, for 

all B e^, 

/x(S) = /x(B,rB < oo) = /x(5,rB < oo). (2) 

Proof. This follows from 

fiiB", Tb = oo) = lim i^iB" n'dB^n---n ^""-^B") 

n— »oo 

= lim i^i-dB" n 'd'^B" n • • • n ^""B") = h{Tb = oo), 

n— »oo 

and similarly for Tg. □ 

In other words, Tb < oo and Tg < oo, fi-a.e. on B. This is used in proving: 
Theorem 1. // the measure /j, on (fi, ^) is preserved by 'd, then, for all A,Bg , 

11b{A) := / MB{A)dn = [ 1(Tb < oo)dii, (3a) 
Jb J a 

]1b{A) := I MBiA)dn = [ 1(Tb < oo)d/x. (3b) 
Jb J a 

Proof. We only need to show the first identity. 

/ MB{A)dn = n{AnB) + y" fi{B n ^-^B" n • • • n ^-""B" n ??-M) 
= n{AnB) + Y^ i^i'd^'B n ^"-^B" n • • • n n A) 

n>l 

= n{A n 5) + ^ fi{TB =n,An B") 



= n{Ar\B) + fi{AnB'',TB < oo) = h{A,Tb < oo), (4) 

where the Poincare recurrence formula (and more precisely its consequence that fj,{A fl 
B,Tb = oo) = 0) was used to obtain the last equality. □ 

Proposition 1 (strong invariance). If the measure fx on {'Tl,^) is preserved by •&, then, 

its restriction fi{- B) on some B E is preserved by •d'^^ and by 'd'^^ , i.e., for all 
A,Be^, 

n{B n = n T^-^^A) = n{A n B). 

Note: The terminology strong invariance is by analogy to the strong Markov property. 
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Proof. Since, due to the Poincare recurrence, ij{B D A n (fi — ^b)) = 0, we have 

oo 

^l{B n d-^^A) = ^ n ^9""^, Tb = n) 

n=l 
oo 

n=l 
oo 

= "^fi{Bn ^-^B" n • • • n i?-^"-^)^^ n n B)) 

n=l 
oo 

= fii'&'^B n n • • • n nAnB) 

n=l 
oo 

= ^ fi{fB = n,AnB) = h^Tb <oo,AnB) = ^{A n B), 

n=l 

where the latter equahty again follows from the Poincare recurrence The second 

assertion is proved in the same manner. □ 

Proposition 2. If the measure /x on (0,, £/) is preserved 'd, then, for all B € £/ , the 
measures /xb(-), /Ib(-); defined by (j3a|) . (|3bp . respectively, are also preserved by 

Proof Note that 

Mb{^-^A) - Mb{A) = l^-T^^ - lA. 
Using this and Proposition [T] we obtain 

Hb{^~^A) - UBiA) = [ MBi^'^A)dn- [ MBiA)dfi = niBn^-^'^A) - n{BnA) = 0. 
Jb Jb 

□ 

Some remarks: 

(i) Since Mb{^) = Tb, Mb{^) = Tb, we have, from Theorem [H 

/ TBdfi = fi{fB < oo), / TBdfj, = ij,{Tb < oo). (5) 
Jb Jb 

Thus, if /i = P is a probability measure and if E denotes integration with respect to P, 
then 

ETbIb = P{Tb < oo) < 1. 
If, in addition, P{B) > then P{fB < oo) > P{fB <oo,B) = P{B) > and so 
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E{Tb I B) 



P{B I Tb < oo) 

where, as usual, E{Tb \ B) = -^^Jy^- This is slightly more general than Kac' formula 
(Kac (1947)). Similar formula holds, of course, for E{Tb \ B): 

^^^^ ' = P{B\TB<ooy 
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(ii) Since Tb > I, Tb > I, ([5]) implies that: 

fi{B) < ^(Tb < oo) a ^(Tb < oo). 

This leads to the following equivalences: 

^(B) > fi{TB < oo) > <^ ^(Tb < oo) > 0. 

Indeed, if I-l{B) > then //(Tb < oo) > from the last inequality. Conversely, if fJ.{B) = 
then dH) shows that /^(Tb < oo) = 0. 

(iii) The function B i— > ^_b(j4) can be thought of as a pre-capacity. Indeed, let 

^'(w) := {^-^uj,^-^uj,...} 

and consider it as a random set. Then 

HB{A)=PiA,^nB^0) 

is the pre-capacity functional of the random set ^ (see Molchanov (2005).) We avoid using 
the terminology capacity because there no topological properties of ^ are introduced. An 
interesting problem would be to investigate properties of the function fJ.B{A) jointly in 
A,B. 

(iv) Theorem [1] and Proposition [2] should of course be linked to the cycle formula of 
Palm calculus, and Proposition [T] to the invariance of the Palm measure. The main point 
here is that within this discrete time setting, there is no need to invoke the general theory 
(Baccelli and Bremaud (2003)). 

(v) Some results do not require the invertibility of ■(?. For instance, fJ,{B) = ^{B,Tb < 
oo) = fJ'{B,[J^^'^'&~'^B) holds for any ^-preserving measurable map "& (Lemma [1]). How- 
ever, the main formulae ()3ap -(j3b [) that exhibit the "duality" between forward and backward 
iterates of do require invertibility. On the other hand, even without using Theorem [1] 
and Propositions [Dill we can show that the measure i^BiA) := fi{A,TB < oo) satisfies 
I'Bi'&A) = UBiA) directly. To do this, note that {Tboi? < oo} = B U {Tb < oo} and write 

fi{'dA,TB < oo) = fi{A,TBo'& < oo) 

= f^{{A \B)U{AnB), BU{Tb < oo}) 
= fi{A \ B,Tb < oo) + fi{A n 5, Tb < oo) 
= fJ.{A,TB < oo). 

This, incidentally, gives a second proof of Proposition [2j 
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3 Uniqueness in Markov chains 



Suppose that P = \pij] is a stochastic matrix on a countable state space S = {a, b,c, . . . . . .}, 

i.e. 

Pi,j>0, ^Pj,fe = l, iJeS. 
kes 

Assume that it is 

(i) irreducible (each i communicates with each j in S) , 

(ii) positive recurrent (starting from some i the expected return time to i has finite 
expectation) . 

These properties depend entirely on the matrix P. It is classical that: 

Theorem 2. If (i) and (ii) hold then there is a unique probability it on S such that vrP = n. 

We wish to show this by using the idea developed in the previous section. 

Proof of existence It is uniqueness that is novel here. Existence of such a tt is imme- 
diately answered by the "cycle formula": Let {Xo,Xi, . . .) be a realisation of the Markov 
chain with transition probability matrix P. Fix some state b, let 

tb := inf{n > 1 : X„ = 6}, 

and define the probability TrC*) on S by 

7rW(a) = :^^^^^^^^, aeS, 

where Eb is expectation conditional on Xq = b. That this tt^^^ is an invariant probability 
measure (satisfies vrWp = is standard (see, e.g. Bremaud (1999)). It is important to 
note that tt^^^ depends entirely on the stochastic matrix P only. □ 



Proof of uniqueness To show uniqueness, we work at the level of sequences, i.e. with 
the space Q. = S^, whose elements are denoted by a; = {iOn,n £ ^), equipped with the 
cylinder cr-algebra =e/. We consider the natural shift 

1? : (rai-^ u;„) (rai-^ tOn+i), 

which is obviously j2/-measurable and invertible. We are thus in the setup of the earlier 
section. Consider a probability vr on satisfying ttP = tt, and let P be the probability 
measure on (5^,i/) defined by 

P{{u en-. Um = im,---,i^n = in}) = '^{im)Pim,im+i ' ' ■Pin-i,in^ 

ijn, ■ ■ ■ ,in ^ S, m,nEZ, m < n. (6) 
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Consider also the random variables 

Xn{uj) := ujn, o; € Jl, n G Z. 

Under P, the sequence (Xn) is a Markov chain with transition probability matrix P. 
Clearly, the measure P is preserved by and, by Proposition [21 so are the measures 



Pb{-) = E1bMb{-) = [ MB{-)dP, 
Jb 



where Tb,Mb are given by (fTa| - ()lbp . for any B ^ £/. Fix some b & S, and consider the 
set 

B = {uj £0, : ujQ = b}. 

Observe that 

ib{uj) := inf{n > 1 : = 6} = Tb{uj), := inf{n > 1 : U-n = b} = Tb{uj)- 

By Theorem [H 

PBiA)=ElBMB{A)=P{A,TB<oo) = P{A,'{b<oo), Ag^/. (7) 
By (i) and (ii) we have P{th < oo) = 1, P(t5 < oo) = 1, and so d?]) yields 

Pb{A) = PiA), AG£/, 

and EIbTb = 1- Therefore, 

PiA) ^ F.(A) ^ E,MA) ^ ^ E,j:'ijH,^..A) ^ 

So, if we pick 

A := {uj : UJQ = a}, 

we conclude that 7r(a) = tt^^^o) for all a € S*. Thus, an arbitrary invariant probability 
measure vr must be equal to the specific measure vr^^^; whence the uniqueness. □ 



Remarks: 



(i) The last argument directly proves that 

Eb ElCo l(^n = a) _E, El=o 



Ehib Ecic 



the so-called exchange formula of (discrete-index) Palm theory (see also Konstantopoulos 
and Zazanis (1995)). 



(ii) Only the existence proof used the Markov property. The uniqueness proof was at 
the level of stationary processes. 
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(iii) In essence, uniqueness follows from the following two facts: 

• Unique determination of the Palm measure: thanks to the Markov setting considered 
here, the Palm law of a cycle starting from a given state until the chain returns to 
this state is uniquely determined by the transition matrix; 

• Slivnyak's inverse construction: this construction shows that the stationary law of a 
point process is fully determined by its Palm measure. (See Slivnyak (1962).) 

Again, the main point here is that there is no need to invoke the general theory. 

(iv) The same argument can be used to show the weaker result: 

Theorem 3. Suppose that (ii) holds (every state is positive recurrent) Let S = Ui>iSi, 
be the decomposition of S into its irreducible components. Let bi € Si, for all i > 1. Then 
every probability -n on S such that ttP = tt is a convex combination of the measures tt^^^K 

4 Uniqueness in Harris chains 

The method explained above can also be applied to yield a proof of uniqueness for the 
invariant probability measure of a positive Harris recurrent chain. 

A Markov process (X„) with values in a Polish space {S, S^) and transition kernel 

K{x,-) = PAXi e •) 

is called Harris recurrent or, simply, Harris chain (Asmussen (2003)) if it possesses a 
recurrent regeneration set R £ S. This means that 

(i) 

Px{iR < oo) = 1, xeS, 

where 

tR := inf{n G N : X„ G i?}; 
(ii) there is a probability measure A on {S, S^) , an e > 0, and i eN, such that 

K'^(x,-)>eX{-), xeR, 

where 

K^{x,-) = PAXee-). 
The chain is called positive Harris recurrent if, in addition to (i) and (ii) we also have 
(iii) 

ExtR < oo. 



8 



where, as usual, Ex denotes expectation with respect to P\{-) := X{dx)Px{-). 

We here give a proof of the following: 

Theorem 4. A positive Harris recurrent chain possesses a unique invariant probability 
measure. 

Note that this theorem is proved in the paper of Athreya and Ney (1978) by different 
methods and only in the case i = 1. There is a substantial difference between the i = 1 
and I > 1 cases in that the cycles defined by the iterates of the stopping time t (see (IllD 
below) are not independent. 

Proof of Theorem^ Existence is standard (see Asmussen (2003)) and requires con- 
struction of the chain on a suitable probability space. We repeat the construction here. 
In addition to the chain, we consider a sequence (Cn) of i.i.d. Bernoulli random variables 
taking values 1 or with probability e or 1 — e respectively. Informally, whenever Xn S R 

K^(x •) — eA(-) 

distribute Xn+£ according to A if = 1 ov according to ^- if (n = 0, and, 

conditional on (X„, X„_|_^), distribute {X^+i, ■ ■ ■ , Xn+i^i) by respecting the given Markov 
kernel. Otherwise, if X„ R, then ignore C„ and continue the chain as usual. Formally, 
we define an ^-th order Markov chain (X„, (^„) with values in x {0, 1} via the following: 
Let G{dxi, . . . ,dx£_i\x,y) be the conditional distribution of {Xi, . . . , X^^i) given that 
Xo = x,Xi = y, i.e. 

I ^ - K{x,dxi) ■ ■ ■ K{xi_2,dxi-i)K{xi^i,dy) 

where the integration in the denominator is with respect to the variables x'^, . . . , x'f^_^ and 
the ratio is to be understood as a Radon-Nikodym derivative with respect to y. Then let 

P{Xn+i G dXi, I <i < i \ Xn = xXn = Ol) 

\{dxi)G{dxi, . . . ^dxi„i\x^xi), if x G i?, a = l 

KHx,dx^)-e\{dx,) ^^^^^^^ .,dx,.,\x,xe), ifxGi?, a = (9) 
K(x,dxi) ■ ■ ■ K{x£-i, dx£), otherwise 
and finally require that, for all n, 

P{Xn+i G dxi, Ci = ai,l<i<£ \ X^, Cm, m < n) 

= p{ai)---p{ae) P{Xn+i £ dxi,l < i < £ \ Xn,Cn), (10) 

where G {0, 1} and p{0) := e, p{l) := I — e. 

It is easy to see that (Xn) is a realisation of the Harris chain with the given transition 
kernel K, and that (Cn) is an i.i.d. sequence; the two sequences are dependent. 

Consider 

t := inf{n : Xn-i G R, Cn^e = 1}, (11) 
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(so that Xi has distribution A) and define 



vr(.) := 



Ext 



(12) 



It is now standard to check that 7r(-) is an invariant probabihty measure for the chain 

To prove uniqueness, we shall again consider the same construction defined by ([8]), 
dl]) and (jlOp . and, in addition, we shall assume that the chain is stationary and therefore 
defined over the index set Z. Specifically, our probability space is Q = {S x {0,1})^, 
equipped with the natural cylinder a-algebra £/. A typical element of O is denoted by 
^ = {{xn^Cn)!^ S Z)- The shift is again the natural one: 

t} : {n^ (x„, Cn)) 1-^ (ni-^ (x„+i, Cn+i))- 

The probability measure P on (Q, £/) is such that it makes the coordinate process an 
i-th. order Markov chain with transition kernel defined through ([8]), ([9]) and (jlOp . and is 
invariant under (Thus, we have created a setup {il., P), as in Section [21 where P 
plays the role of /U and, here, -P(il) = 1.) We now prove that there can be only one such 
P. To this end, let 

B ■= {uj = (x, C) e : e R, C~t = !}• 
By our assumptions, P{Tb < oo) = 1, P{Tb < oo) = 1. By Theorem [H 



PeiA) = E1bMb{A) = P{A,Tb < oo) 
and EIbTb = 1- But 



P{A), Ae 



E 



Pb{A) 



E1bMb{A) 
EIbTb 



t-i 



l(i?"w e A) I G R, = 1 



,n=0 



t-1 



i?A^l(^"u;G A) 



n=0 



E[t I € R,C-e = 1] 



since, by construction, P{C\x-i G R,C-i = 1) = P\{C) for any C in the a-algebra 
generated by {ujn,n > 0). Taking A := {lo = (x,C) € ^ ■ xq G ■} we conclude that 
any ??-invariant probability measure P that preserves the given Markovian structure must 
have a marginal given by (jl2p . This proves uniqueness. □ 

Final note: The proof of uniqueness, again, uses arguments that do not rely on the 
Markov property. As such, it would be worth exploiting it further in stochastic scenaria 
with absence of Markovian property. 
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